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ABSTRACT 


One  aspect  of  management  of  ADP  (Automated  Data  Processing) 
in  the  Federal  Government  is  system  documentation.  "Good  system 
documentation  is  the  backbone  to  the  success  of  any  automated 
system,  for  only  through  complete  and  thorough  documentation  can 
the  user  fully  understand  and  utilize  a  system's  capability." 
(Kaplan89,  page  29].  The  system  reviewed  is  the  Naval  Air 
Logistics  Command  Management  Information  System  (NALCOMIS).  The 
review  is  approached  from  the  user's  aspect.  The  review  of  this 
system  involves  the  level  of  ability  required  for  the  user  to 
understand  and  properly  use  the  functionality  provided  by  the 
system,  the  documentation  requirements,  and  the  problems 
associated  with  the  user's  interface  to  the  system.  Many 
problems  exist  with  computer  manuals  that  contained  an 
inappropriate  level  of  technical  detail,  style,  format,  and 
content  [Kaplan89,  page  30].  This  review  will  demonstrate  the 
positive  and  negative  aspects  of  the  user  documentation  and 
recommendations  or  suggestions  to  improve  the  documentation  for 
the  user's  benefit. 
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SYSTEM  INTRODUCTION 


NALCOMIS  was  on-line  and  functioning  in  August  1990,  the 
system  is  approximately  two  years  old.  This  review  is  based  on 
the  Phase  II  system  which  is  implemented  at  the  Intermediate 
Maintenance  Activities  (IMA)  and  Supply  Support  Centers  (SSC). 
Specifically  reviewed  was  the  Cecil  Field  Aviation  Intermediate 
Maintenance  Detachment  (AIMD). 

NALCOMIS  provides  the  capability  to  manage  Naval  aviation 
maintenance  and  material  requirements,  providing  detailed 
processes  to  enter,  collect,  process,  store,  review,  report  and 
interface  data  required  by  the  organization.  These  processes  are 
in  support  of  engine,  component,  and  support  equipment  repair, 
material  requisitions,  repairable  stock  management,  awaiting 
parts  management,  direct  support  material  control,  supply  file 
maintenance,  personnel  assignment  and  deployment,  subcustody  of 
equipment,  and  utilization  of  resources  at  the  IMA/SSC  levels. 
These  functions  are  integrated  into  one  system  sharing  a  common 
data  base  to  avoid  redundancy  of  functions  and  related  data 
between  the  organizations,  improving  overall  communication  and 
response  time  associated  with  material  readiness  [ NAVMASS089 , 
page  3 ] . 
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SYSTEM  FUNCTIONS  AND  DOCUMENTATION 


The  user  interface  displays  a  series  of  menu  driven  screens 
and  conversations  that  allow  manipulation  of  the  NALCOMIS  data. 

It  is  interactive  on-line  processing,  requiring  initial  log-in 
and  password  to  enter  the  system.  Initially  a  series  of  menu's 
will  guide  the  user  to  the  subsystem  and  functional  areas  within 
the  subsystem  desired,  controlling  the  user's  access  to 
conversations.  The  more  experienced  user  can  go  directly  to  the 
conversation  desired  using  the  appropriate  conversation  code  to 
access  the  conversation.  The  user  is  only  allowed  to  perform 
conversations  that  are  authorized  through  the  user's  password. 

To  the  novice  user,  this  system  is  quite  complicated.  Even 
though  it  is  menu  driven  the  user  must  know  where  to  go  and  how 
to  get  there  before  entering  the  system.  The  system  is  not  self- 
descriptive.  There  is  an  on-line  help  menu  that  describes  the 
fields  and  the  kind  of  information  that  goes  into  each  field. 

The  on-line  help  is  an  on-line  description  of  available  options 
or  explanations  of  how  the  system  works.  This  can  provide  a  fast 
and  convenient  form  of  help  for  users  stuck  in  a  specific 
situation  or  data  entry  field.  [Mischo89,  page  32]. 

One  of  the  biggest  drawbacks  of  this  on-line  help  system  is 
that  it  doesn't  tell  the  user  where  to  get  the  information 
required  or  where  the  information  goes  once  it  is  added  to  the 
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system.  "On-line  documentation  can  be  both  more  immediately 
responsive  and  more  frustrating.  This  is  especially  true  of  help 
screens,  which  often  answer  the  wrong  question."  [Bills89, 
page  34].  The  information  found  using  the  on-line  help  is  very 
similar  to  what  is  found  in  the  User  Manuals.  It  refers  to  other 
manuals  and  documents  unrelated  to  the  system  for  codes  needed  to 
complete  the  data  entry. 

Effective  on-line  help  information  is  difficult  to  get 
except  for  the  simplest  problems.  As  with  printed  documentation, 
the  key  seems  to  be  proper  indexing  properly  presented  to  the 
user.  "Since  the  on-line  option  implies  immediate  answers,  the 
time  needed  to  get  to  and  work  through  on-line  help  indexes  may 
be  less  acceptable  to  the  user  than  time  spent  on  manuals." 
[Bills89 ,  page  34].  In  the  NALCOMIS  system,  the  help  function  is 
very  easy  to  use.  An  'H'  is  placed  in  the  Action  Code  block  on 
the  menu  screen  and  the  screen's  function  will  be  described.  If 
help  is  desired  for  a  particular  field  on  the  screen,  the  menu 
choice  is  put  in  the  selection  block  and  'H'  is  entered  in  the 
Action  Code  block.  On-line  help  is  handy  to  the  novice  user  who 
knows  very  little  about  the  system.  It  does  not  help  the  more 
experienced  user  that  is  trying  to  determine  why  the  system  will 
not  accept  a  valid  code  or  why  it  will  not  validate  on  a  field 
necessary  to  continue.  A  context  sensitive  help  screen  describes 
the  screen  it  is  linked  to  and  all  the  options  possible  for  that 
screen.  "If  the  problem  is  with  the  mechanics  of  the  specific 
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screen,  this  information  is  good  and  handy.  However,  if  the 
question  is  "Why  didn't  this  search  work?",  the  help  screen  seems 
incredibly  unresponsive."  [Bills89,  page  34]. 

"A  manual  with  a  clear  and  immediately  apparent 
organization,  a  good  index,  and  complete  instructions  help  the 
user  get  into  and  around  the  application  faster  and  more  easily." 
[ Bills89 ,  page  34].  There  are  several  written  manuals 
accompanying  the  NALCOMIS  system.  These  manuals  are  constantly 
updated  whenever  a  software  change  or  "patch"  is  made.  The  User 
Manuals  describe  the  system,  screen  by  screen  and  field  by  field. 
There  is  a  table  of  contents  to  show  the  order  of  the 
descriptions.  The  manuals  are  written  in  user  friendly  terms  and 
are  easy  to  understand.  They  are  organized  in  the  order  of  the 
screens  displayed.  "The  surest  way  documentation  can  fail  is 
with  a  poorly  written  index.  A  good  index  can  make  badly  written 
documentation  at  least  usable.  A  bad  index  can  make  well-written 
documentation  a  source  of  frustration."  [Bills89,  page  34]. 
NALCOMIS  manual's  biggest  downfall  is  the  lack  of  an  index.  The 
manuals  are  big  and  bulky  and  difficult  to  use  to  obtain  a  quick 
answer  to  a  question.  The  lack  of  an  index  or  glossary  for  a 
quick  look-up  of  the  user's  question  make  the  manual's  unwieldy. 
These  manuals  are  not  distributed  to  every  terminal,  therefore 
not  used  frequently.  Documentation  has  to  be  clear,  available, 
and  easy  to  assimilate  in  order  to  assure  confidence  in  a  system. 
Production  Control  and  the  System  Administrator  refer  to  the  user 


manuals  occasionally  when  needed. 


The  Conversation  manual  is  used  extensively  to  solve 
problems  by  the  System  Administrator.  The  conversation  manual 
consists  of  detailed  descriptions  of  the  conversations  used  to 
manipulate  the  data.  Each  conversation  is  described  in  modular 
form  by  name,  code,  type,  description,  action,  mailbox  messages 
created,  hardcopy  notice  created  and  system  interface.  It 
describes  the  screens  and  reports  affected  by  each  conversation. 
An  example  screen  is  displayed  with  the  code  lengths  and  types 
described,  the  purpose  of  the  screen  and  the  valid  entries.  This 
helps  the  user  to  debug  the  system  when  codes  will  not  validate 
or  the  system  doesn't  perform  as  expected. 

"Most  manuals  help  the  user  get  started  (often  with 
difficultly)  providing  an  introduction  to  the  functions,  an 
overview  of  elementary  operations  and  a  warning  against  errors. 
Few  manuals  stay  apace  of  users  as  they  gain  knowledge  of  the 
system,  or  need  shortcuts,  advanced  features  or  available 
customization."  [Mischo89,  page  31].  In  an  interview  with  the 
systems  administrator,  her  chief  complaint  about  the 
documentation  is  that  the  level  of  information  available  through 
the  manuals  is  insufficient.  She  is  not  sure  whether  the 
documenters  did  not  include  the  information  needed  because  it's 
too  technical,  they  don't  think  the  user  needs  the  information  or 
they  are  unaware  the  information  is  needed.  "Too  often  the 


presentation  obscures  the  information,  it  is  inaccessible,  or  it 
is  even  missing.”  [Mischo89,  page  32].  An  example  given  by  the 
system  administrator  was  a  of  a  report  output.  Only  through 
hours  of  experimentation,  changing  information  in  the  data  base 
and  printing  reports  was  she  able  to  discover  which  columns  or 
sections  of  information  affected  the  results  of  other  columns  of 
data.  The  derivation  of  information  especially  for  reports  is 
not  included  in  the  manuals. 

Other  problems  with  the  system  includes  validations.  While 
traversing  through  the  screens  the  system  will  not  let  the  user 
continue  until  certain  fields  are  validated.  There  is  no 
documentation  explaining  why  these  particular  fields  need  to  be 
validated  and  why  it  doesn't  validate  if  certain  combinations  of 
codes  are  used.  An  example  quoted  by  the  System  Administrator 
was  the  use  of  a  modular  part  number  instead  of  the  individual 
part  number.  The  System  rejected  the  code  only  giving  a  message 
that  the  number  was  not  on  file.  It  took  intense  research  to 
discover  that  the  part  desired  had  a  separate  identification 
code.  This  was  the  number  required.  The  documentation  assumed 
the  user  knew  this  information  and  did  not  spell  it  out  anywhere. 
Vendor  supplied  documentation  is  limited  to  the  "how”  of  system 
operation  and  rarely  addresses  the  "why" .  As  a  result  vendor 
documentation  fails  where  the  user  needs  it  the  most.  [Mischo89, 
page  34], 
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When  the  System  Administrator  encounters  a  bug  in  the 
system,  that  she  can  not  fix  or  resolve  through  the 
documentation,  NAVMASSO  (Navy  Management  Systems  Support  Office) 
is  contacted  via  telephone  or  message,  through  the  chain  of 
command,  as  to  the  problem.  They  will  come  back  with  a 
resolution  or  if  it  cannot  be  resolved  without  a  software  change 
will  develop  a  "work  around"  for  the  user  until  a  software 
"patch"  is  developed.  This  source  of  information  to  the  user  is 
slow  and  bureaucratic.  System  complaints  go  through  the 
Commander,  Naval  Air  Atlantic  (COMNAVAIRLANT) ,  who  controls  all 
sites  utilizing  NALCOMIS  on  the  East  Coast,  to  NAVMASSO  and 
resolutions  are  sent  back  through  the  same  channels.  This  method 
of  communication  has  positive  and  negative  aspects.  On  the 
positive  side,  if  one  site  has  a  problem  the  solution  is 
distributed  to  all  sites  at  the  same  time  eliminating  duplication 
of  effort.  On  the  negative  side,  the  response  is  slow,  the 
system  problems  are  handled  on  a  priority  basis,  but  the  process 
to  fix  a  problem  or  crisis  can  be  extremely  long. 


The  Systems  Administrator  receives  the  new  documentation  for 
a  "work  around"  and  develops  a  step  by  step  guide  on  the 
procedures  needed  to  use  the  "work  around".  She  distributes 
these  instructions  to  all  work  centers  that  may  need  to  know  the 
procedures  of  that  particular  "work  around".  This  works  very 
effectively  and  with  these  detailed  step  by  step  instructions, 
anyone  familiar  with  the  system  can  make  the  corrections. 
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Unfortunately  there  is  not  more  documentation  in  this 
format.  There  is  a  process  to  make  recommendations  to  the  system 
developers  and  documentation  writers  on  format,  usability  and 
nice  to  have's,  but  this  process  takes  time  and  is  done  in  order 
of  priority. 

When  new  software  patches  are  installed,  the  system  is 
brought  down  until  installation  is  complete.  For  the  annual 
major  patches,  a  team  is  sent  to  the  site  in  order  to  train  the 
users  on  the  system  changes.  For  minor  patches  documentation  is 
sent  to  the  user  with  the  updated  revisions.  "Software  changes 
that  don't  get  reflected  in  the  documentation  or  that  exist  as 
"add  in"  pages  are  difficult  to  use.  The  information  needed  must 
be  available  in  the  manual."  [Mischo89,  page  33], 

Other  than  the  annual  updace  training  there  is  not  a  formal 
training  schedule  for  the  novice  user  to  learn  the  system.  The 
novice  user  is  trained  on  the  job,  on  a  one  to  one  basis  with  an 
experienced  user  until  he/she  understands  the  system.  Only  after 
he  is  very  familiar  and  understands  the  system  will  he  be  given  a 
password  to  enter  the  system  interactively  and  be  able  to  make 
changes  to  the  data. 

A  tutorial  manual  is  a  step  by  step  task  oriented  walk¬ 
through  of  how  the  application  runs  written  with  the  assumption 
users  know  nothing  about  the  application.  For  a  novice  just 
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learning  the  job,  the  tutorial  is  the  most  essential  piece  of 
documentation  [Mischo89,  page  33].  NALCOMIS  does  not  have  a 
tutorial  or  step  by  step  guide  to  explain  the  system  to  the 
novice  user.  This  is  the  largest  drawback  of  the  documentation 
for  this  system.  The  training  is  “seat  of  the  pants"  trial  and 
error,  the  training  packages  that  are  available  are  usually  just 
for  corrections  to  problems  that  have  been  reworked  and  replaced 
with  a  software  "patch" . 

"When  small  work  groups  are  affected  by  poor  documentation 
they  can  pool  informal  knowledge  and  learn  by  trial  and  error." 
[Mischo89,  page  31].  The  System  Administrator  described  weekly 
meetings  incorporating  all  the  work  centers  and  Supply  Support 
Center  supervisors  involved  with  NALCOMIS  to  work  out  problems, 
discrepancies  and  any  bugs  in  the  system  locally.  Desk  Top 
guides  or  quick  reference  materials  with  alphabetical  listings 
and  short  descriptions  of  all  commands  are  non-existent.  The 
development  of  locally  used  reference  manuals  and  desk  top  guides 
would  make  the  system  easier  to  use  and  would  save  the  system 
administrator  a  lot  of  time  and  energy  spent  answering  other  less 
experienced  workers  questions. 
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RECOMMENDATIONS 


Effective  documentation  is  composed  of  both  style  and 
content.  There  are  many  writing  styles  for  documentation. 
Overall,  the  documentation  for  NALCOMIS  is  very  good.  Each 
screen  is  described  and  each  field  is  explained.  The  manuals  use 
examples,  actual  screens,  and  the  explanations  are  easy  to 
understand.  The  manuals  are  bulky,  an  easy  fix  would  be  to  add 
section  dividers  which  would  enable  an  easier  access  to  each 
section  of  each  manual.  Other  negative  aspects  include  not 
enough  information  in  the  manuals  to  answer  complicated  questions 
and  the  lack  of  step  by  step  procedures  or  instructions  which 
would  make  the  documentation  more  usable.  "Searching  for 
instructions  not  in  the  manual  is  frustrating  and  discourages 
further  use  of  the  documentation."  [Mischo89,  page  34].  A 
solution  to  the  problem  is  to  develop  guides  at  the  local  level, 
where  each  site's  procedures  are  a  little  different.  The  vendor- 
supplied  documentation  doesn't  always  meet  local  requirements. 

The  needs  of  two  kinds  of  users  should  be  addressed.  The 
initial  user,  the  person  who  has  to  learn  the  system  from  scratch 
needs  a  beginning-to-end  simple-to-coraplex  explanation  of  the 
functions  and  pitfalls.  The  experienced  user  is  focused  on  a 
specific  task  or  tasks.  Trained  by  the  vendor  (NAVMASSO)  or 
another  staff  member,  he  now  needs  a  description  of  routine 


11 


functions  and  a  source  of  help  only  when  unusual  problems  arise. 
[Bills89,  page  34]. 

Quick  references  and  desk  top  guides  would  help  to  put  the 
more  useable  information  at  the  fingertips  of  the  user.  A  local 
experienced  user  should  be  assigned  to  develop  these  guides  at 
the  local  level . 

A  PC  based  tutorial  would  help  the  novice  user  to  become 
familiar  with  the  system  before  he/she  was  required  to  interact 
with  the  database.  This  could  also  be  in  the  form  of  a  ’’training 
manual  for  first-time  users  that  begins  with  an  overview  of  the 
system.  In  this  overview,  its  various  components  are  identified 
and  a  step-by-step  approach  to  using  the  system  is  provided." 
[Matthews89,  page  36].  This  set  of  exercises  will  help  the  user 
master  frequently  encountered  situations  and  transactions.  This 
step  by  step  tutorial/training  manual  could  have  actual  working 
examples  and  solutions  to  each  system  process.  The  system  should 
be  organized  so  that  only  certain  processes  are  performed  by 
certain  people,  the  work-center  user  would  only  attempt  the 
tutorials  needed  for  his  job,  where  as  the  Production  Control 
user  may  go  through  examples  of  all  screens  used  in  production 
control.  This  would  give  the  user  some  hands  on  experience  with 
built  in  solutions  and  the  ability  to  learn  the  system  without 
destroying  important  data.  Some  recommendations  for  a  tutorial 
for  the  novice  user  include  tutorial  guides,  videotapes  and  PC 
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based  tutorials  that  are  self-instructional  with  each  lesson  or 
chapter  building  on  the  previous  one.  "The  tutorials  should 
present  introductory  instructions  only  and  make  no  attempt  to 
comprehensively  cover  all  system  features."  [Anderson89, 
page  36]. 

The  last  recommendation  would  be  to  incorporate  into  the 
NALCOMIS  data  base  the  information  taken  from  other  instructions 
and  documentation  to  derive  the  codes  necessary  to  use  the 
NALCOMIS  system.  At  the  present  time  the  help  screens  will  refer 
the  user  to  an  instruction  or  directive  to  locate  the  correct 
code.  If  these  codes  and  descriptions  were  located  in  the  data 
base,  it  would  save  the  user  a  lot  of  time  and  enhance  the  value 
of  the  system. 
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SUMMARY 


The  objectives  of  NALCOMIS  is  to  track  repairables,  monitor 
time  spent  ordering,  repairing  and  total  time  to  fix  parts.  The 
system  is  set  to  track  parts  through  the  life  of  that  part, 
storing  information  in  order  to  determine  exactly  what  and  where 
the  part  is  at  any  point  and  time.  The  system  accomplishes  this 
goal.  It  is  a  well  planned  system.  The  system  documentation  is 
suitable  for  the  needs  of  the  system,  but  not  necessarily  the 
needs  of  the  user.  "User  documentation  must  focus  on  the  users, 
helping  them  to  overcome  their  apprehension  and  resistance  to  the 
new  and  the  strange.  It  should  build  their  confidence  in  their 
ability  to  use  the  system  easily  and  efficiently  by  providing 
them  with  easy-to-follow  instructions."  [Webb91,  page  41].  As 
with  any  system  there  are  always  improvements  that  can  be  made. 
The  documentation  in  this  system  can  also  be  improved.  The  main 
goal  is  to  keep  the  system  functioning  properly.  Improving  and 
updating  documentation  should  be  considered  at  the  same  time,  the 
system  will  run  smoother  and  the  user's  will  be  more  satisfied. 
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